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INTRODUCTION 


Advanced control technology poses a difficult task for the authorities 
faced with specifying airworthiness flying qualities requirements — and for 
the manufacturers who must comply with and anticipate these requirements. 
Requirements for advanced civil transports employing this technology must be 
carefully framed, such that public safety is ensured and technological ad- 
vances in civil aviation are not discouraged. It is no secret that exces- 
sively complex and overstringent requirements discourage innovation, while 
clear and flexible requirements (for example, those that give credit for re- 
liability in systems) encourage development and .advances in technology. 

The specification of flying qualities requirements Involves considera- 
tion of the complete pilot-airframe-systems loop, the task, and the environ- 
ment. Figure 1 suggests the complexity of this job; many of these advanced 
civil configurations tend to be large and flexible and dependent on complex 
control systems for enhancement of stability, control effectiveness, and con- 
trol feel characteristics over enlarged flight envelopes, and for numerous 
automatic control modes. The result is a greatly increased emphasis on fail- 
ure effects that degrade flying qualities. Key questions being faced in- 
clude: How good must the flying qualities be in the failure condition? 

Which failures and combinations must be demonstrated? And how must they be 
demonstrated? 

French and British authorities, in preparing for Concorde SST certifica- 
tion, authored a new form of flying qualities requirements that rely heavily 
on probabilistic analyses (TSS Part 3, ref. 1). In TSS 3, the required 
standard of flying qualities varies according to the likelihood of the flight 
condition occurring, and thus considers the wide range of flight phases, 
system failure effects, and atmospheric environment. Although it is being 
applied to Concorde by European authorities and some features of the method 
have been utilized in U.S. military specifications, the TSS 3 approach has 
met with mixed reactions among the U.S. civil aviation community because of 
concerns over the practical implementation of the method. 

Since 1969, an ongoing NASA/FAA research program has used the Ames 
Flight Simulator for Advanced Aircraft (FSAA) in the development of certifi- 
cation criteria for supersonic cruise aircraft. NASA, FAA, industry repre- 
sentatives, and British and French airworthiness authorities are participat- 
ing in this program. The question of proper accountability of failures has 
arisen on numerous occasions. These experiences have brought to a focus the 
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need to review the present treatment of failure cases in the requlr^nents and 
to examine some of the questions associated with implementation of the TSS 3 
type of concept. 

This paper, which reports on the findings to date from a continuing study 
of the subject, comprises the following: a review of the treatment of fail- 

ure cases in various flying qualities requirements; a description of methods 
used and relevant lessons learned from recent Autoland certification programs 
as an example of applied probability procedures; a discussion of uncertain- 
ties about the TSS approach; and finally (because these procedures indicate 
an increasing reliance on simulation methods) , a description of three recent 
experiences with marginal configurations that demonstrate the potential sig-^ 
nlflcance of elements sometimes omitted from simulation tests. 


CURRENT TREATMENT OF FAILURE CASES IN VARIOUS 
FLYING QUALITIES REQUIREMENTS 


Aircraft flying qualities requirements deal primarily with controllabil- 
ity, stability, and handling characteristics. Civil and military require- 
ments were reviewed for the maimer in which failure cases were covered, the 
amount of flying qualities degradation allowed, the conditions under which 
failures were to be assessed (for example, introduction of atmospheric ef- 
fects), and methods for demonstrating compliance. (As used throughout this 
paper, the term "failure" includes malfunctioning as well as failure to 
function; degraded system performance below specified tolerances represents 
a failure to function properly.) Documents reviewed included Federal Avia- 
tion Regulations applicable to transport category airplanes (FAR 25, ref. 2), 
Tentative Airworthiness Standards for Supersonic Transports (TASST, ref. 3), 
Industry recommendations (AIA committee report, ref. 4 and SAE Aerospace 
Recommended Practice 842B, ref. 5), Franco-Britlsh Concorde TSS Standards 
(TSS, ref. 1), and U.S. military specification (MIL-F-8785B, as described in 
ref . 6) . 


Federal Aviation Regulations - FAR 25 and TASST 

For orientation, an outline of FAR 25 is shown in figure 2. Flying 
qualities requirements are contained in "Subpart B - Flight," which is 
further broken down into topic headings. Although flying qualities are 
closely interrelated with many performance requirements (many of which in- 
volve engine failure conditions) , this discussion is primarily concerned 
with those items indicated by an arrow, and the related paragraphs in "Sub- 
part D - Design and Construction" and "Subpart F - Equipment." 

Failure Cases in FAR 25- Philosophy towards treatment of failures has 
undergone significant change in recent years. For years, about the only 
multiple failure cases were two-engine-inoperative control requirements and 
a requirement that/ the airplane be controllable with all engines inopera- 
tive. Only single control system failures were considered. In April 1970, 
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Amendment 25-23 incorporated a number of changes into FAR 25 dealing with 
system failures and introducing the consideration of multiple failures. 
Stability augmentation systems and automatic systems were dealt with specif- 
ically. Some of the new requirements came from the tentative SST require- 
ments and were recognized to be generally applicable and needed because of 
the increasing dependence on more complex systems of the new generation of 
subsonic transports. The example shown in figure 3 illustrates the present 
treatment of control system failures. As indicated, FAR 25.671 requires the 
capability of continued safe flight and landing after any single control 
system failure or after any combination of failures not shown to be extremely 
improbable. 

Current FAA interpretation of the terms "probable," "improbable," and 
"extremely improbable" is shown in the sketch below. 
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Failure cases in the other aircraft systems must also be analyzed under 
"Subpart F - Equipment." In addition to requiring the capability of con- 
tinued safe flight and landing after any failure condition not extremely 
Improbable, FAR 25.1309 requires that the systems and associated components 
be designed so that 

"the occurrence of any other failure conditions which 
would result in injury to the occupants, or reduce the 
capability of the airplane or the ability of the crew 
to cope with adverse operating conditions is improbable." 

In addition, nearly all requirement sets contain catchall paragraphs 
which are in general terms, but provide evaluation pilots basis for rejection 
of unsatisfactory situations not covered specifically. An example of this is 
FAR 25.143 which states 

"(a) The airplane must be safely controllable and 
maneuverable during - (1) takeoff; (2) climb; (3) level 
flight; (4) descent; and (5) landing, (b) It must be 
possible to make a smooth transition from one flight 
condition to any other without exceptional piloting 
skill, alertness, or strength and without danger of 
exceeding the limit-load factor under any probable 
operating conditions (including the sudden failure of 
any engine) . " 
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The phrase "under any probable operating conditions" is certainly sub- 
ject to interpretation as including failure cases. 

Failure Cases in TASST- In TASST (ref. 3), the FAA presented tentative 
airworthiness standards for study, trial application, and comment during the 
detail design and prototype phase of supersonic transport development. A 
number of changes were proposed in this document other than those already 
discussed, including requirements to cover automatic and manual trim system 
malfunctions, additional two-engine-inoperative controllability and maneuver- 
ability requirements, and extended "flutter, deformation and fail-safe cri- 
teria" to consider combinations of failures not shown to be extremely im- 
probable. In the "Stability" section, it was recognized that areas of flight 
(for example, supersonic cruise) may exist with operational requirements such 
that the use of reliable automatic flight control systems could be accepted 
in lieu of the demonstration of classic static stability, provided the loss 
of automatic flight control would not result in unsafe handling character- 
istics. 

The "Structures" section of TASST is closely related to flying qualities. 
Paragraph 25.301(e) states that 

"For supersonic aircraft, loads must be determined 
within the design flight envelope considering the ef- 
fects of stability augmentation and automatic flight con- 
trol systems, including probable failures and changes in 
systems characteristics which can be expected in service. 

All malfunctions and failures of these systems must be 
considered under FAR 25.671 and FAR 25.1309 within the 
normal flight envelope except those shown to be extremely 
improbable." 

Careful consideration of the complete pilot-aircraft-systems loop and 
the environment appear very important in satisfying this requirement. Also 
note that this introduces the assessment of failure effects outside the 
normal flight envelope (see sketch) . The application of flight simulation 
techniques would appear to be essential for this task. 

NORMAL FLIGHT ENVELOPE - 
PROBABLE AND IMPROBABLE 
FAILURES (> 10-9 per hour) 

MUST BE CONSIDERED IN 
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Turbulence and Flexibility Effects- In the introductory discussions to 
both the "Controllability and Maneuverability" and the "Stability" sections 
of TASST, it was recognized that the effects of turbulence on the pilot en- 
vironment should be evaluated. In addition, it was pointed out that the 
structural flexibility and stability characteristics of supersonic trans- 
ports will undoubtedly aggravate the pilot environment problem. Flight ex- 
perience with the XB-70 and F-12 series aircraft lends considerable weight to 
these statements — as have some piloted simulator experiences with large 
flexible configurations to be described later. It is very likely that many 
of the advanced transport designs will exhibit greater flexibility than cur- 
rent subsonic transports and, as will be shown, the effects on handling 
characteristics in failure-mode operations can be very significant. 


Industry Recommendations 

AIA Study Group Proposals- In 1970, a special project group represent- 
ing the Aircraft Industries Association (AIA) published the results (ref. 4) 
of a study to guide the modernization of the Federal Air Regulations. In 
reference 4, proposed "modernized" requirements are presented as a set of 
safety standards generally applicable to all transport aircraft types. 

These standards describe basic characteristics of the aircraft system that 
must be achieved to ensure safe operation. In addition, means for showing 
partial or complete compliance with the individual standards are Included. 

Two fundamental requirements formed the foundation for all the standards 
proposed; 

"1, The aircraft must respond to commands of the con- 
trolling intelligence in a consistent manner and 
with the precision appropriate to the task. 

2. Probable subsystem failures must not result in 

conditions likely to be catastrophic due to human 
inability to cope with them." 

These modernized standards specify three modes of operation (manual, 
cOTimand, automatic), conveying clearly that the controlling intelligence is 
not always considered to be the human pilot. They state further that if man 
is the controlling intelligence, he should be considered a subsystem of the 
total aircraft system. In this way, the standard dealing with operation 
following failures accounts for failures of human origin in addition to other 
subsystem failures. This standard states, 

"Operation following probable failure of any subsystem 
that affects flight safety shall not unduly restrict 
flight operation after corrective action is taken. 

The degree of restriction permitted shall be inversely 
related to the probability of failure." 

It then presents requirements related to the ability to take corrective 
action, either by the crew or by automatic means. The acceptable means of 
compliance deal more specifically with the failures, and include paragraphs 
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that parallel FAR 25.671(c) and 25.672(c) (fig. 3). 


SAE Design Criteria- Reconmended design criteria for handling qualities 
of civil transport aircraft (SAE ARP 842B, ref. 5) differ in character from 
the safety requirements described previously. These criteria represent ad- 
visory design information as defined by the SAE and were originally modeled 
after the format of the military specifications of the early sixties. These 
criteria appear to have avoided the use of probability terminology and in- 
clude consideration of single and dual control system failures. 

"... Following the [single] most critical failure in 
the [power or boost] flight control system, the planned 
flight may be completed without a significant degrada- 
tion of flying qualities. . . . Following the second 
most critical flight control system failure, it shall 
be possible to complete the flight, after takeoff, to 
a suitable airport from the V 2 transition to enroute 
climb to cruise to a safe landing with the most critical 
engine inoperative at the most critical phase of flight." 

They further state that failure of any artificial stability system or 
powered-actuated trim system should not result in an unsafe flight condition 
Some of the quantitative criteria (for example, lateral control) are rede- 
fined for the failure cases to accept degraded capability. 

The significance of aeroelastic effects is recognized in paragraph 
2.1.7 of ARP 842B which states 

"Since it can be expected that aeroelastic effects 
will play an important role in supersonic transport 
design, it should be clear that all requirements for 
flying qualities are applicable to the elastic air- 
frame." 


Franco-British TSS 3 

General Description and Objectives- A new approach to flying qualities 
requirements was developed by the French and British airworthiness authori- 
ties in preparation for the certification of supersonic transports, Concorde 
in particular. First published in 1969 as TSS 5 and since changed to TSS 3 
(ref. 1), these requirements are currently being applied to Concorde. Their 
most significant feature is the extensive use of probabilities and systems 
analysis methods in defining the minimum acceptable flying q\ialities for a 
given flight situation, considering the flight phase, aircraft configuration 
failure state, and environment. The severity of the requirement is directly 
related to the probability of occurrence of the flight situation. This con- 
cept has since been utilized in the British Provisional Airworthiness Re- 
quirements for Civil Powered-Lift Aircraft (ref. 7) and in modified form in 
the current U.S. military specification MIL-F-8785B. 
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This approach provides the following significant advantages: 

1. a more systematic and complete coverage of all likely flight con- 
ditions, whereas past methods have tended to be limited to antici- 
pated critical regions 

2. consideration of atmospheric environment effects in a more complete 
manner 

3. a running assessment of the relative risk level throughout the design 
and development phases for a new aircraft, which provides insight for 
design modifications 

4. a method for defining those cases that can be eliminated from demon- 
stration because of the low probability of occurrence. 

The TSS standards are Intended to provide the same safety levels for 
supersonic transports as for subsonic airplanes introduced into service at the 
same time. These objectives include the following: "For all airworthiness 

causes the total probability of Catastrophic Effects should be Extremely Re- 
mote [<10~^ per hour of flight], and the total probability of Hazardous Ef- 
fects should be remote [<10“5] or Extremely Remote." (See table 1 for defini- 
tion of terms.) Akin to FAR 25, these objectives state that "No single Fail- 
ure or combination of failures not considered Extremely Improbable shall re- 
sult in a Catastrophic Effect." They further require that "Remote Failures 
shall not result in Hazardous Effects" and that "Recurrent Failures shall re- 
sult only in Minor Effects." 

The TSS 3 requirements are categorized into three groups, corresponding to 
the accident causes attributed to flying qualities: (1) handling - a workload 

consideration, (2) maneuverability, and (3) involuntary exceedance of airplane 
limits caused by disturbances due to failures or atmospheric conditions. Var- 
ious specific criteria are included which, depending on the probability of oc- 
currence of a given "state" (categorized as frequent, occasional, exceptional, 
and non-exceptional) , must be satisfied. There are also a number of require- 
ments, based on judgment and experience, which require demonstration regard- 
less of the estimated probability of occurrence. 

Theoretical Application- Figures 4 and 5 illustrate theoretical applica- 
tion of the TSS 3 concept. (Reference 9 points out that practical applica- 
tion requires many simplifying assumptions, although little information on 
these assumptions has been found in the literature.) First, the various 
possible flight "tasks" and their associated probabilities of occurrence per 
flight are defined. As shown in figure 4, a task is defined by four primary 
elements plus the secondary workload: (1) the flight subphase, for example, 

localizer capture; (2) state of the atmosphere; (3) state of the aircraft, 
which includes possible failures; and (4) flight technique. Elements 1 and 
4 represent lists prepared by the applicant while elements 2 and 3 represent 
four-dimensional matrices. The probability Pji of a given task per flight 
is then calculated from estimates of the probabilities of (1) performing a 
given subphase per flight, (2) encountering a given atmospheric state during 
the subphase, (3) having a given aircraft state during the subphase, and (4) 
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using a given flight technique. 

Figure 5 represents the author's interpretation of a method described in 
TSS 3 for showing compliance with the general handling requirement. Pilot 
evaluation of a given task identifies a class of difficulty C, which is then 
converted to the probability that a pilot will not be able to accomplish 

the workload. The probability of a handling incident during a given subphase 
per flight is determined by summation over the classes of difficulty of the 
product of P^j and Pjj for that subphase. The total probability of a hand- 
ling incident per flight is computed by summation over all the subphases that 
make up a flight. For partial compliance, this total probability must then 
be less than a safety index, which has been defined as an acceptable risk 
level. 

TSS 3 states that the demanded safety level is to be demonstrated by a 
limited number of flight tests proposed by the applicant. (Justification for 
tests omitted is also required.) The majority of these are to be conducted 
in calm air or low turbulence. Compliance with requirements for flight in 
turbulence are to be demonstrated by a limited number of flight tests, sup- 
ported by theoretical studies and simulator tests. 

From this brief description, it is clear that numerous questions can be 
raised regarding the practical application of this approach and that consid- 
erable simplifications are needed. In the next section, simplifications are 
described which have been made in the application of a similar procedure in 
the U.S. military specification. While all the uncertainties are not laid 
to rest, discussions in the following sections address many of the expressed 
concerns, and point the way for continuing work. 


U.S. Military Specification (MIL-F-8785B) 

Similarity with TSS Concept- MIL-F-8785B (presented with background in- 
formation in ref. 6) serves dual roles as design requirements and as evalua- 
tion criteria. At a 1971 AGAKD meeting, a paper (ref. 9) was presented com- 
paring the TSS 3 concept and MIL-F-8785B. It concluded that they are basic- 
ally the same in intents and goals, although one distinction was made: in 

addition to assuring that there will be no limitations on flight safety due 
to deficient flying qualities, MIL-F-8785B demands that mission effective- 
ness will not be compromised. Similarity of the two criteria is not coin- 
cidental; discussion following presentation of the AGARD paper acknowledged 
the significant contributions made by M. Wanner, representing the Service 
Technique Aeronautique of France and a strong advocate of the TSS 3 concept, 
during the preparation of MIL-F-8785B. 

A number of simplifying assumptions have been made to permit practical 
application of MIL-F-8785B, including: 

(1) No probability assessment is made for aircraft mass and mass dis- 
tribution. A probability of 1 is used for all points in the 
envelope. Thus, probability of state of the aircraft is dependent 
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on failure probabilities only. 


(2) No attempt is made to estimate the probability of the state of the 
atmosphere. The required flying qualities are associated with the 
state of the airplane. (A number of specific flying qualities re- 
quirements must be met with specified turbulence conditions, how- 
ever . ) 

(3) The probability of being in a given area of the flight envelope has 
been assumed equal to 1, due to inability to specify this value. 

"Levels” of Flying Qualities- Three levels of flying qualities are de- 
fined in MIL-F-8785B, as shown in table 2. Cooper-Harper pilot ratings gen- 
erally associated with the three levels are also shown. Exceptions to these 
relationships exist, however. For example, level 3 flying qualities for a 
landing task would correspond to a pilot rating no poorer than 6.5 (requires 
adequate performance; see fig. 6). 

The minimum required flying qualities are defined separately for air- 
plane normal states and airplane failure states. For airplane normal states, 
level 1 flying qualities are required within the operational flight envelope, 
and level 2 within the service flight envelope (fig. 7). For airplane fail- 
ure states, the probability of encountering level 2 flying qualities must be 
less than 10“^ per flight within the operational flight envelope and the 
probability of encountering level 3 flying qualities must be less than 10“'^ 
per flight in the operational flight envelope and less than 10“^ in the 
service flight envelope. 

Theoretical Compliance Procedure- Figure 8 illustrates the procedure 
outlined in MIL-F-8785B for determining theoretical compliance with the fail- 
ure state requirements. Airplane failure states that have a significant ef- 
fect on flying qualities are first identified and the corresponding probabil- 
ities of encounter per flight are computed, based on the longest flight dura- 
tion to be encountered during operational missions. The degree of flying 
qualities degradation associated with each airplane failure state is deter- 
mined in terms of levels as defined in the specific requirements. The most 
critical airplane failure states are then determined (assuming the failures 
are present at whichever point in the flight envelope being considered is 
most critical in a flying qualities sense) , and the total probability of 
encountering level 2 flying qualities in the operational flight envelope due 
to equipment failures is computed. Likewise, the probability of encounter- 
ing level 3 flying qualities in the operational flight envelope is computed. 
The computed values are then compared with the requirements. 

Concept Recommended for Civil Airworthiness Application- Many of the 
military specifications were originally recommended by Cornell Aeronautical 
Laboratory, Inc. (now the Calspan Corporation) under contract to the Air 
Force Flight D 3 mamics Laboratory. In 1973, Calspan completed a review of 
the "Flight" subpart of the Yellowbook (Tentative Airworthiness Standards 
for Powered Lift Transport Category Aircraft) for the FAA. The final report 
(ref. 11) proposed that the Yellowbook be revised to a new format based on 
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many of the ideas used in the military specification and in the British 
Provisional Airworthiness Requirements for Civil Powered-Lift Aircraft 
(ref. 7). 


General Observations 

Based on review of the various requirements, several observations can be 
made. All elements of the aviation community have acknowledged the need for 
increased attention to failure effects and have made the transition from 
single failure to multiple failure philosophy. The prediction of system fail- 
ure probabilities and their effects has become a significant factor in flight 
certification of aircraft employing stability augmentation, automatic, and 
powered control systems. For aircraft employing active controls technology 
to full advantage, the system failures and effects analyses are even more 
important. This results in a growing need for close integration of the sys- 
tems and flying qualities disciplines. Present U.S. certification practice 
appears to treat the systems and flying qualities evaluations somewhat 
separately, with the effects of failures often defined by analytic means in 
the systems studies. While this procedure may have served adequately in the 
past, the foregoing observations suggest that they will, at the very least, 
require reexamination for future applications. 

General recognition is apparent that atmospheric effects (e.g., turbu- 
lence) can influence an airplane’s handling characteristics significantly and 
should be considered, although the method of including this is loosely de- 
fined. The high cost of flight testing, the large number of cases to be 
evaluated, the desire to assess in specified atmospheric conditions, and at 
marginally safe conditions can be expected to Increase the reliance on 
piloted flight simulators for much of this work. 


FAILURE CASE ANALYSES IN AUTOLAND CERTIFICATION 


Systems safety analysis procedures used in recent Autoland certifica- 
tion programs represent current examples of the application of probability 
procedures to the certification of total airframe-systems combinations, in- 
cluding consideration of atmospheric effects. Because of the close relation- 
ship with the evaluation concepts previously discussed, the procedures used 
in the Category IIIA automatic landing programs for the McDonnell Douglas 
DC-10 and the Lockheed L-1011 (refs. 12-14) were reviewed and relevant find- 
ings are noted. 


Procedures 

The procedure described in ref. 13 appears to be generally representa- 
tive of the programs for both airplanes. The certification process, which 
represents the final cycle of studies made in the design and development 
phases, used progressive simulation and testing, as indicated in figure 9, 
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in order to minimize the amount of flight testing required. The first step 
was the use of high-speed repetitive-operation simulation methods to accom- 
plish the millions of landings required for establishment of the low prob- 
ability results in a reasonable time period. In the second phase, several 
thousand simulated landings were made using the actual flight hardware com- 
puters. The hydraulic control systems hardware ("iron bird") was then added 
to the simulation in order to pick up effects of any hardware imperfections. 
Finally, a minimal number of flight test demonstrations (on the order of a 
hundred) were made to verify the high end of the performance probability 
curves. Some of the simulated failure effects were verified by inserting 
failures into the autopilot during actual approaches. Each of these phases 
was used to verify the results of the preceding phase. 

Environmental conditions for these simulations included turbulence and 
wind shear, with levels specified in FAA Advisory Circular 20-57A. In the 
DC-10 program, key performance characteristics of the sensors, analog com- 
puter, and mechanical controls were varied between simulation runs within the 
normally expected ranges using a Monte Carlo sampling routine (ref. 12). 

Not evident in the procedure just described is the reliability and 
safety analysis, a considerable task consisting bf an integrated combination 
of several kinds of analyses and computer simulation techniques. This ex- 
tensive process is described in detail in ref. 12. Suffice to say that it 
involved identifying all possible single and multiple faults in the system 
and their effects, eliminating all single faults that were hazardous, and 
establishing that no multiple fault in the system having a probability of 
occurrence greater than 10“9 per landing was hazardous. 


Relevant Findings 

Integrated Programs Necessary- Ordinary numerical reliability analyses 
were recognized at the outset to be inadequate for fully assessing Autoland 
system reliability and safety. Because of the basic system complexity, the 
airhorne-ground systems interfaces, and the numerous pilot-aircraft inter- 
faces, integrated programs of laboratory testing, computer analyses, and 
simulation were found necessary. 

Design Guidance Provided- Certification considerations began with the 
design phase. As the system design evolved, the reliability and safety 
analyses provided continued assessment of compliance and identified areas 
requiring design modification. Consideration of failure effects signifi- 
cantly influenced the design of many other aircraft systems, for example, 
electrical supply. 

Multiple Failure Analysis Found Manageable- The multiple failure analy- 
sis appeared at first to be an almost impossible task, requiring the com- 
bination of all possible failures in all possible sequences and analyzing the 
result. This task became manageable by first defining what was hazardous and 
then working backward to find all combinations of faults that could produce 
the event. 
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Definition of Atmospheric Disturbances Needed- Atmospheric disturbance 
effects can become primary design factors. In some flight tests, for example, 
a condition not anticipated to be critical — a quartering tailwind> — was found 
to be serious. Other flight test experience has indicated that present spec- 
ified wind shear values may be inadequate. The potential significance of 
such disturbances makes accurate definition of the atmosphere essential. 

Broad-based Engineering Judgment Necessary- A fundamental merit and a 
hazard of the probability approach are revealed in this quotation from ref. 

13: 


"The probability approach to analysis seems, from 
experience, to have great merit in that the necessity 
to calculate very low probability numbers forces on 
the analyst a discipline that makes him study the sys- 
tem in greater detail. The danger in the approach is 
that the analyst may place too much emphasis on the 
techniques he has developed, and lose sight of the 
many assumptions implied in these techniques. In 
short, there is a danger of placing implicit belief 
on the accuracy of a calculated- number. This danger 
can be avoided by the use of highly skilled engineers 
who are capable of understanding system and aircraft 
operation as well as the detailed working of the cir- 
cuits to be analyzed." 


Concluding Observations 

Although the Autoland systems safety analysis procedures described here- 
in appeared extremely cumbersome at the outset, in actual practice they be- 
came manageable — while providing significant payoffs in terms of design 
guidance and Improved safety. It should also be noted, however, that while 
the effort involved in an Autoland certification program is undoubtedly 
large, the effort appears small when compared to the total effort required 
in rigorously applying the procedures of TSS 3 to an advanced transport air- 
craft over its entire flight envelope. The number of cases to be considered 
for Autoland is limited: the Autoland process is concerned primarily with 

the final few minutes of flight, and the controlling intelligence can be 
mathematically modeled more readily than can the human pilot. 


UNCERTAINTIES REGARDING THE TSS APPROACH 


While the potential advantages of the TSS 3 type of approach have been 
shown to be very significant (partially verified by the Autoland experience) , 
numerous questions and uncertainties have been raised regarding its practi- 
cal implementation. These can be grouped under the following headings: (1) 

reliance on probability methods and reliability prediction, (2) size of the 
evaluation matrix, (3) use of the pilot rating scale, (4) definition of the 
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atmospheric environment, and (5) use of simulation methods. In the following 
discussion, each of these has been addressed In order to provide some In- 
sight Into these topics, to dispel some fears, and to Indicate where further 
work Is needed. 

Reliance on Probability Methods and Reliability Prediction- Concern has 
been expressed with regard to the ability to define some of the required 
probabilities, such as the flight subphase, pilot technique, and atmospheric 
environment. Conservative engineering estimates of the first two should be 
possible with careful study. Definition of the probability of a given atmos- 
pheric environment appropriate to a given subphase requires more research 
(to be discussed later) . 

In reliability and safety analyses, there Is always the danger of 
"blind faith" In the calculated number. As pointed out In TSS 3 and In refs. 

8 and 15, these methods are used as an aid, not as the sole criterion; It Is 
essential that they be combined with good engineering judgment and experience. 
The practical limitations of a given method must be taken Into account and 
experience with other aircraft In service must be factored Into the total 
assessment. For example, the present safety assessment of redundant systems 
goes far beyond the failure analysis by considering possible effects of er- 
rors by the crew and maintenance personnel, as well as the effects of events 
outside the aircraft which could affect more than one channel at a time. 
Redundant systems are checked for common faults to ensure, for example, that 
both electrical systems are not routed through a common wiring bundle or 
under galleys and toilets, or that lines from both hydraulic systems are not 
supported by a common bracket. 

A common question asks how probability values of the order of 10“^ per 
flight hour can be estimated with confidence. Reference 16 points out that 
this Is exactly the reason for the philosophy that no single failure can 
create a catastrophic flight condition. The period of proof -testing re- 
quired to prove this failure rate would be Impractical. However, the In- 
dividual failure rates of Interest In multiple-failure analyses, of the order 
of 10“^, can usually be estimated with reasonable confidence. It is also 
intended that critical system failure records be kept on new aircraft enter- 
ing service over the Initial period of operation to verify the reliability 
estimates. 

Evaluation Matrix- The matrix of conditions requiring evaluation under 
the procedure described in TSS appears awesome. However, considerable sim- 
plification appears possible and merits continued study. Also, in practice, 
the number of failures to be Investigated normally turns out to be a manage- 
able number (ref . 8) . The fault analysis usually shows a limited number of 
ways a system can malfunction following a variety of single and multiple 
faults. Many can be discarded because the result is not serious or the 
probability of occurrence is clearly satisfactory. 

Use of the Pilot Rating Scale- Concern has been expressed over making 
a pilot rating scale a part of legal regulation, to be used in determining 
the minimum safety level of an airplane. Questions faced whenever the pilot 
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rating scale is used become especially significant when it is the minimum 
safe boundary being defined. Typical questions are: What pilots are to do 

the rating, how many, how to extrapolate from the flight test situation to 
the operational one, etc. These questions and others are worthy of careful 
study and resolution. Considerable worthwhile discussion on many of these 
issues is contained in ref. 10. Many of the questions raised, however, are 
not unique to the TSS procedure, but are equally applicable to the present 
evaluation process where the subjective opinions of the airworthiness pilots 
are key factors in defining the acceptability of a given airplane. 

Definition of the Atmospheric Environment- Definition of the significant 
elements of the atmospheric environment and associated probabilities is an 
area receiving considerable attention in the U.S. and in Europe, and justifi- 
ably so. The influences of atmospheric disturbances become especially trouble- 
some as flying qualities are degraded and the workload approaches the satura- 
tion point. 

Many of the turbulence models currently being used in simulation studies 
have been tailored to match power spectra measurements. Other concepts are 
being studied. For example, recent work in the U.K., stimulated by Autoland 
experience, is investigating the use of discrete gust patterns (ref. 17), and 
work is continuing in this country under NASA sponsorship at the University of 
Washington and elsewhere to develop "non-Gauss ian" models. Also, an investi- 
gation devoted to verification or improvement of present methods for modeling 
aircraft response to turbulence appears worthwhile. 

Simulation Methods- The preceding discussions leave little doubt that the 
use of simulation methods will play an increasingly key role in the design, 
development, and certification of advanced transport aircraft. The applica- 
tion of simulators to the certification demonstration process must not be 
approached naively, but with appreciation for the limitations of these methods 
and for the degree of fidelity (math model, pilot station layout, visual dis- 
play, motion, etc.) required for specific tasks. Representation of the appro- 
priate workload level, for example, is an important factor in evaluating mini- 
mum safe handling qualities. 

For a safety assessment as defined in TSS 3, development of this simula- 
tion capability early in the design phase, with progressive updating of the 
airplane model and the pilot/pilot station interface, appears essential. Ac- 
quisition of data for improvement of simulation fidelity must be factored into 
layout of early flight tests. Accurate representation of failure annunciators 
and warning devices must be incorporated as they are defined, as they are im- 
portant elements in the evaluation of a proposed system's acceptability. 


RELATED SIMULATION EXPERIENCES 


The preceding discussions lead to the conclusion that the final acceptance 
of many aircraft failure states may be based largely on simulator evaluations 
(and engineering judgment) . Three recent simulation experiences have empha- 
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sized factors which, with more stable configurations, might have been consid- 
ered of secondary importance, but became critical components requiring accurate 
representation in the simulation of marginal configurations. The first empha- 
sizes the significance of turbulence effects, the second indicates the impor- 
tance of motion cues in critical tasks, ami the third demonstrates control 
limitations that can be Imposed by structural mode effects. 


Turbulence Effects 

In 1972, parallel studies were conducted on two simulators to investigate 
the SAS-f ailed approach and landing of delta-wing transports (refs. 18 and 
19) . Three research test pilots performed ground-based evaluations on the 
NASA/ Ames six-degree-of -freedom Flight Simulator for Advanced Aircraft (FSAA) , 
followed by flight evaluations on the USAF/Calspan Total In-Flight Simulator 
(TIFS) , both shown in figure 10. In a matrix of twenty test configurations, 
seventeen were unstable longitudinally. The primary task was an ILS approach 
under IFR conditions, breakout to VFR conditions at 91-m (300-ft) altitude, 
visual approach and landing. A series of approaches with added tasks Included 
crosswind approach, glide-slope error correction, localizer error correction, 
and moderate (0.91 m/sec or 3.0 ft/sec rms) turbulence. Of these, the turbu- 
lence task proved to be the most critical, although the turbulence intensity 
used was not uncommon (probability of encountering turbulence of 0.91 m/sec or 
greater is on the order of 0.1 to 0.3). 

Pilot rating data from both investigations is shown versus a divergence 
parameter, time to double amplitude of angle of attack T2a» figure 11. 

Pilot ratings from the FSAA study are shown by the shaded band, with the scat- 
ter primarily attributable to interpilot variation. Values of T 2 of 6 sec 
and greater were found to be acceptable for the emergency case. As T£ de- 
creased (divergence rate increased) below this level, pilot ratings show that 
handling characteristics deteriorated rapidly. 

Initial examination of the TIFS pilot rating data showed considerable 
scatter due to the varying turbulence intensities encountered during the 
flights. In analyzing the data, Calspan used measurements of the actual tur- 
bulence environment to compensate the pilot rating data for each configuration. 
These results are shown in figure 11 for gust intensities of 0.46 m/sec (1.5 ft/ 
sec) and 0.91 m/sec (3.0 ft/sec). Although the difference between the two 
levels of turbulence intensity appears small, the differences in subjective 
evaluation were significant. 


Motion Effects 

An investigation was conducted at Ames recently to identify the role of 
cockpit vertical acceleration cues in the landing task (ref. 20). A piloted 
simulator having very large amplitude vertical motion (24 meters total travel) 
was utilized in a test series in which the fidelity ("washout") of the vertical 
acceleration reproduction was deliberately varied over a wide range, represent- 
ing simulators with varying amounts of available vertical travel. The external 
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visual scene was provided by a black and white uncollimated TV monitor. The 
airplane simulation represented a large sweptwiiig business jet transport. 

Three levels of static longitudinal stability were simulated, corresponding to 
15 percent static margin, neutral, and 5 percent unstable static margin. 

The results indicated that vertical motion cues were utilized in the land- 
ing task and were particularly important in the simulation of aircraft with 
marginal longitudinal handling qualities. Figure 12 shows a measure of landing 
performance, altitude rate at touchdown, plotted against the motion washout 
filter natural frequency 0 )^. The corresponding vertical travel requirements 
are shown along the top scale. The data indicate that the effect of motion was 
relatively inconsequential for landing of the stable configuration with good 
flying qualities, although an oscillatory tendency was observed without motion. 
However, with the configurations having marginal longitudinal handling quali- 
ties, significant degradation was apparent in achievable performance as the 
motion was constrained (and thereby distorted). At values of com of 1.0 and 
above, divergent flight path oscillations were common and touchdowns were es- 
sentially uncontrolled in many landings. 


Structural Mode Effects 

In another simulation program in which a very large flexible aircraft was 
represented complete with structural modes, a very significant degradation in 
flying qualities resulted from the pilot station motions caused by fuselage 
bending. Evaluations of the completely unaugmented airplane without motion 
and body bending resulted in pilot ratings of 5.0 - 5.5 (fig. 6). With motion 
and body bending, the structural modes were easily excited and the pilots were 
unable to use the sharp pulse Inputs (in pitch) normally used for control of 
an unstable airplane. This prevented the use of effective control techniques 
and yielded a pilot rating of 9. 


Recommendat ions 

These examples have illustrated the fact that careful attention must be 
devoted to defining the simulation requirements for a given task. A high de- 
gree of sophistication is often required in evaluations of marginal cases if 
confidence is to be placed in the results. Practical design and evaluation 
procedures will very likely, by necessity, rely on simplified simulations (very 
limited motion, no structural mode representation, etc.) for the bulk of the 
work. It is emphasized that verification testing of critical cases should be 
planned in simulation facilities which provide a high fidelity of the total 
task presentation. 


CONCLUDING REMARKS 


Advanced transport aircraft designs have become increasingly dependent on 
complex flight control systems in order to improve their flight characteristics 
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In this report, various civil and military flying qualities requirements have 
been reviewed with regard to their treatment of failure cases and considera- 
tion of atmospheric environment effects. There appears to be common acceptance 
of the philosophy that no single failure should create an unsafe flight con- 
dition, nor should any combination of failures that are not extremely improb- 
able. Although consideration of atmospheric environment effects in handling 
assessments is required, the method for doing this is often ill-defined. 

There is an increasing need for an orderly procedure for combining the 
systems analyses (reliability and fault analyses) with the flying qualities 
evaluation process, taking the likely atmospheric states into account. Such 
a procedure can aid in the achievement of design economies and a level of 
safety equivalent to that of current transports; this is a challenging task 
since the contribution of system failures to catastrophic effects is at pres- 
ent a very small proportion of the total. Review of the probability-based 
procedures described in the Anglo-French TSS 3 shows that additional develop- 
ment effort is needed to simplify implementation. Simplified procedures de- 
scribed in the U.S. military specification and lessons learned from recent 
Autoland programs appear useful for continuing studies devoted to this pur- 
pose. 


In order to minimize flight testing and to enable evaluation in specified 
atmospheric conditions and hazardous failure cases, simulation techniques will 
be used extensively in such procedures. Some recent simulation experiences 
emphasize that turbulence effects significantly influence the pilot evalua- 
tions of marginal configurations, and that evaluation of such conditions some- 
times requires more accurate representations of the pilot/aircraft interface 
(pilot station motions, etc.) than are provided in many current engineering 
simulations. 
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Table 1. Definition of probability terms (ref. 8) 


Failures 

Effects 

Type 

Definition 

Recurrent failures 

Minor effects 

Can readily be counteracted by crew and 
may involve: 

(a) small increase in work load. 

(b) moderate degradation in perfor- 
mance or handling. 

(c) slight modifications to the per- 
missible flight envelope. 

Remote failures 

Major effects 

May produce: 

(a) significant increase in crew work 
load. 

(b) significant degradation in perfor- 
mance or handling characteristics. 

(c) significant modification of the 
permissible flight envelope. 

but will not remove the capability to 
continue a safe flight and landing 
without demanding more than usual skill 
on the part of the flight crew. 

Extremely remote 
failures 

Hazardous 

effects 

These effects may be more than major 
providing that the overall risk of 
catastrophe is extremely improbable, 
taking into account likely crew action. 

Extremely improbable 
failures 

Catastrophic 

effects 

Resulting in fatalities. 


Reourvent. (Frequency of occurrence up to about lO"'^ per hour of flight.) 
Expected to occur from time to time in the life of an airplane. 

Remote. (Of the order of 10~5 to 10“^ per hour of flight.) May happen a few 
times during the total operational life of a type of aircraft. For ex- 
ample, a remote failure includes failure of two engines in one flight. 

Extremely Remote. (Not expected to occur more often than 10”^ per hour of 
flight.) Unlikely to occur during the total operational life of all air- 
craft of a type, but nevertheless has to be considered as being possible. 

Extremely Improbable. So extremely remote that it can be stated with confi- 
dence that it should not occur. 
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Table 2.- Flying qualities levels from MIL-F-8785B 


Level 

Description 

Corresponding 
pilot rating 
(in general) 

1 

Flying qualities clearly adequate for the mission 
flight phase 

1 - 3.5 

2 

Flying qualities adequate to accomplish the 
mission flight phase, but some increase in pilot 
workload or degradation in mission effectiveness, 
or both, exists 

3.5 - 6.5 

3 

Flying qualities such that the airplane can be con- 
trolled safely, but pilot workload is excessive or 
mission effectiveness is inadequate, or both. 
Category A flight phases can be terminated safely, 
and Category B and C flight phases can be 
completed. 

6.5 - 9+ 


ATMOSPHERE 




STATE 

CONFIGURATION 

WEIGHT 

MASS DISTRIBUTION 
STABILITY AND CONTROL CHAR. 



TASK 


Figure 1.- Factors influencing flying qualities 
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FAR 25 AIRWORTHINESS STANDARDS: TRANSPORT CATEGORY AIRPLANES {ref. 2) 


SUBPARTS 

A. GENERAL 

B. FLIGHT 

GENERAL 

PERFORMANCE: RECIP. ENGINE POWERED AIRPLANES 
PERFORMANCE: TURBINE ENGINE POWERED AIRPLANES 



CONTROLLABILITY AND MANEUVERABILITY 



TRIM 



STABILITY 



STALLS 

^ "■ 


GROUND AND WATER HANDLING CHARACTERISTICS 



MISCELLANEOUS FLIGHT REQUIREMENTS 

^ — 

c. 

STRUCTURE 


D. 

DESIGN AND CONSTRUCTION 

MMI 

E. 

POWERPLANT 


F. 

EQUIPMENT 

III 

G. 

OPERATING LIMITATIONS AND INFORMATION 



Figure 2.- FAR 25 outline identifying sections of interest. 


25.181 DYNAMIC LONGITUDINAL. DIRECTIONAL. 
AND LATERAL STABILITY. 

ANY SHORT PERIOD OSCILLATION OCCURRING 
BETWEEN STALLING SPEED AND MAXIMUM 
ALLOWABLE SPEED APPROPRIATE TO THE 
CONFIGURATION . . . MUST BE HEAVILY DAMPED 
WITH PRIMARY CONTROLS (1) FREE AND 
(2) IN A FIXED POSITION. 


25.21(e) 

IF COMPLIANCE WITH THE FLIGHT 
CHARACTERISTICS REQUIREMENTS IS 
DEPENDENT UPON A STABILITY 
AUGMENTATION SYSTEM OR UPON ANY 
OTHER AUTOMATIC OR POWER- 
OPERATED SYSTEM, COMPLIANCE MUST 
BE SHOWN WITH PARAGRAPHS 25.671 
AND 25.672. 


SUBPART D DESIGN AND CONSTRUCTION 


25.671 CONTROL SYSTEMS - GENERAL 

(c) THE AIRPLANE MUST BE SHOWN BY ANALYSIS, TESTS. OR 
BOTH TO BE CAPABLE OF CONTINUED SAFE FLIGHT AND 
LANDING AFTER ANY OF THE FOLLOWING FAILURES OR 
JAMMING IN THE FLIGHT CONTROL SYSTEM AND SURFACES 
. . . , WITHIN THE NORMAL FLIGHT ENVELOPE, WITHOUT 
REQUIRING EXCEPTIONAL PILOTING SKILL OR STRENGTH. 
PROBABLE MALFUNCTIONS MUST HAVE ONLY MINOR EFFECTS 
ON CONTROL SYSTEM OPERATION AND MUST BE CAPABLE OF 
BEING READILY COUNTERACTED BY THE PILOT. 

(1) ANY SINGLE FAILURE . EXCLUDING JAMMING. 

(2) ANY COMBINATION OF FAILURES NOT SHOWN TO BE 
EXTREMELY IMPROBABLE , EXCLUDING JAMMING 

(3) ANY JAM IN A CONTROL POSITION NORMALLY 
ENCOUNTERED . . . UNLESS THE JAM IS SHOWN TO 

BE EXTREMELY IMPROBABLE OR CAN BE ALLEVIATED . . . 

25.672 STABILITY AUGMENTATION AND AUTOMATIC AND 
POWER-OPERATED SYSTEMS 

(b) THE DESIGN . . . MUST PERMIT INITIAL COUNTERACTION 

OF FAILURES OF THE TYPE SPECIFIED IN PARAGRAPH 25.671(c) 
WITHOUT REQUIRING EXCEPTIONAL PILOT SKILL OR STRENGTH. 
BY EITHER DEACTIVATION OF THE SYSTEM. OR A FAILED 
PORTION THEREOF. OR BY OVERRIDING THE FAILURE BY 
MOVEMENT OF THE FLIGHT CONTROLS IN THE NORMAL SENSE. 


(c) IT MUST BE SHOWN THAT AFTER ANY SINGLE FAILURE . . . 

(1) THE AIRPLANE IS SAFELY CONTROLLABLE WHEN THE 
FAILURE OR MALFUNCTION OCCURS AT ANY SPEED OR 
ALTITUDE WITHIN THE APPROVED OPERATING LIMITATIONS 
THAT ARE CRITICAL FOR THE TYPE OF FAILURE BEING 
CONSIDERED; 

(2) THE CONTROLLABILITY AND MANEUVERABILITY REQUIRE- 
MENTS OF THIS PART ARE MET WITHIN A PRACTICAL 
OPERATIONAL FLIGHT ENVELOPE . . . DESCRIBED IN THE 
AIRPLANE FLIGHT MANUAL; AND 

(3) THE TRIM, STABILITY. AND STALL CHARACTERISTICS ARE 

NOT IMPAIRED BELOW A LEVEL NEEDED TO PERMIT CONTINUED 
SAFE FLIGHT AND LANDING. 


Figure 3.- Example of control system failure treatment in FAR 25. 
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1. FLIGHT SUBPHASE 

ELEMENTARY OBJECTIVE 

WITH TOLERANCES 

{e.g. LOCALIZER CAPTURE) 


2. STATE OF THE 
ATMOSPHERE 

• TURBULENCE 
INTENSITY 

• TEMPERATURE 
GRADIENT 

• VISIBILITY 

• MEAN WIND 
ON GROUND 


3. STATE OF THE 
AIRCRAFT 

• MASS 

• MASS DISTRIBUTION 

• SELECTED 
CONFIGURATION 

• FAILURES 


4. FLIGHT 
TECHNIQUE 
CHOICES AVAILABLE 
TO PURSUE SUBPHASE 
OBJECTIVE 


PLUS SECONDARY WORK 

(CHECKLIST, RADIO TRAFFIC, NAVIGATION) 

CONSTITUTE A TASK 


PROBABILITY (TASK) * 

PROS {SUBPHASE PER FLIGHT) x 
PROB (ATMOSPHERIC STATE DURING SUBPHASE) x 
PROB (AIRCRAFT STATE DURING SUBPHASE) x 
PROB (GIVEN CHOICE OF FLIGHT TECHNIQUE) 


Figure 4.- Elements defining the "task" in TSS 3 (refs. 1 & 16). 



SUBPHASE A 


SUBPHASE 6 


SUBPHASE C 


SUBPHASE TABLE 


1 SUBPHASE 

C; INtTIAL CLIMB ] 

TASK 

C 

Py 

Pn 


C3 

10-” 

.90 

2 

C4 

10“’ 

.06 

3 

C4 

10“’ 

.01 

4 

C6 

10-^ 

2 K 10-® 


(EXAMPLE 

ONLY) 


CLASSES OF 
DIFFICULTY 


= 10"” (.90) + 10"^ (.06 + .01) + 10“^ (2 X 10“® + . . . ) 


PROB (INCIDENT/FLIGHT) = i 

- IPg X Pgl 

SUBPHASES 

CLASSES OF 


DIFFICULTY 


< SAFETY 
INDEX 


Figure 5.- One possible method described in TSS 3 (ref. 1) for 
showing compliance with handling requirement. 
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HANDLING QUALITIES RATING SCALE 



Figure 6.- Cooper-Harper pilot rating scale (ref. 10). 


NORMAL STATES 


FAILURE STATES 



PROB (LEVEL 2) 

< 10-2 PER flight 

PROB (LEVEL 3) 

< 10-« PER FLIGHT 


PROB (LEVEL 3) 

< 10-2 PER FLIGHT 


Figure 7.- 


MIL-F-8785B minimum 


flying qualities requirements 
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REPRODUGBILITY of THl 

ORK^IAij ?4fi ^ fOOM 




DURATION OF LONGEST FLIGHT 
DURING OPERATIONAL MISSIONS 



Figure 8.- 


MIL-F-8785B procedure 
with airplane failure 


for determining theoretical compliance 
state requirements. 
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Figure 9.- Progressive simulation and testing in Autoland 
certification (ref. 13). 



TOOL - NAS A/ AMES 
FLIGHT SIMULATOR FOR 
ADVANCED AIRCRAFT 



\ 


Figure 10.- Simulation study of minimum longitudinal stability 
for SAS-f ailed landing. 
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Figure 11.- Pilot rating vs time to double amplitude T2o(, showing effect of 

turbulence intensity (Og is rms value) . Landing task, unaugmented 
delta-wing transport. 


CORRESPONDING TOTAL 
VERTICAL TRAVEL, meters 
24 18 11 6.7 4.0 2.4 




Figure 12.- Vertical motion effects on landing performance (ref. 20). 
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